Dating images from scanned watermarks

ABSTRACT

A method for dating photographic prints comprising scanning a front side of a photographic print forming an image scan; scanning a back side of the photographic print forming a back scan; defining a database of photographic print watermarks, each photographic print watermark comprising a representation of a watermark pattern and an associated date range; analyzing the back scan using a cross-correlation process between the back scan and the watermark patterns in the database of photographic print watermarks to identify a matching photographic print watermark; determining a date range for the image scan responsive to the identified matching photographic print watermark; and associating the determined date range with the image scan.

FIELD OF THE INVENTION

The present invention pertains to the field of image organization andmore particularly to determining a date range for a photograph.

BACKGROUND OF THE INVENTION

Consumers today are switching from film-based chemical photography todigital photography in increasing numbers. The instantaneous nature ofimage capture and review, the ease of use, numerous output and sharingoptions, multimedia capabilities, and on-line and digital media storagecapabilities have all contributed to consumer acceptance of thistechnological advancement. A hard drive, on-line account, or a DVD canstore thousands of images, which are readily available for printing,transmitting, conversion to another format, conversion to another media,or used to produce an image product. Since the popularity of digitalphotography is relatively new, the majority of images retained by atypical consumer usually takes the form of hardcopy media. These legacyimages may span decades of time and have a great deal of personal andemotional importance to the collection's owner. In fact, these imagesoften increase in value to their owners over time. Thus even images thatwere once not deemed good enough for display are now cherished. Theseimages are often stored in boxes, albums, frames, or even their originalphotofinishing return envelopes.

Getting a large collection of legacy images into a digital form is oftena formidable task for a typical consumer. The user is required to sortthrough hundreds of physical prints and place them in some relevantorder, such as chronology or sorting by event. Typically events arecontained on the same roll of film or across several rolls of filmprocessed in the same relative time frame. After sorting the prints theuser would be required to scan the media to make a digital version ofthe images. Scanning hardcopy image media such as photographic prints toobtain a digital record is well known. Many solutions currently exist toperform this function and are available at retail from imaging kiosksand digital minilabs and at home with “all-in-one” scanner/printers orwith personal computers equipped with media scanners. Some mediascanning devices include media transport means, simplifying the task ofscanning hardcopy media. Using any of these systems requires that theuser spend time or expense converting the images into a digital formonly to be left with the problem of providing some sort oforganizational structure to the collection of digital files generated.

The prior art teaches sorting scanned hardcopy images by physicalcharacteristics. However, this grouping would be limited to coarsegroupings of images and a general chronological sequence, which may beinadequate for very large image collections and may not be as useful assmaller groupings. What is needed is a system to rapidly convert largevolumes of hardcopy media images into digital form and to automaticallyorganize them into an improved chronological order and more accurateevent groupings. This will provide a way for a consumer to easily andaffordably obtain a digital version of a hardcopy image collection thatwill be a high quality presentation provided with a meaningful context.

U.S. Pat. No. 6,745,186 entitled “Product and Method for Organizing andSearching digital Images” describes methods of organizing digital imagesby sorting or organizing scanned hardcopy images by physicalcharacteristics including shape, size, cut, texture, border or finish.

U.S. Pat. No. 6,606,411 entitled “Method for Automatically ClassifyingImages into Events” describes methods for separating a group of imagesinto events on the basis of time or date. Long lapses of time are usedas event boundaries. Thus, when grouping images for output products likealbums or for organizing a database of images, the images are grouped bysimilar time stamps.

U.S. Pat. No. 6,636,648 entitled “Albuming Method with Automatic PageLayout” describes methods for laying out album pages on the basis oftime or date and content. What is meant by content in this patent is abasic image analysis that identifies similar colorations such ashistograms.

U.S. Pat. No. 6,351,321B1 entitled “Data Scanning and Conversion Systemfor Photographic Image Reproduction” describes methods for identifyingcamera exposed information such as date/time/exposure conditions ondigitized print images and employing techniques to edit out, crop,enhance, and replace the camera exposed information.

While some of the forgoing patents provide some help in organizingimages, there still is a need to provide further efficient techniquesfor automatically organizing images into further defined groupings.

SUMMARY OF THE INVENTION

The present invention represents a method for dating photographic printscomprising:

a) scanning a front side of a photographic print forming an image scan;

b) scanning a back side of the photographic print forming a back scan;

c) defining a database of photographic print watermarks, eachphotographic print watermark comprising a representation of a watermarkpattern and an associated date range;

d) analyzing the back scan using a cross-correlation process between theback scan and the watermark patterns in the database of photographicprint watermarks to identify a matching photographic print watermark;

e) determining a date range for the image scan responsive to theidentified matching photographic print watermark; and

f) associating the determined date range with the image scan.

This invention has the advantage that the date of a photographic printcan be estimated without a prior knowledge provided by a user. Thisenables the organization of large numbers of photographic images withoutthe need for user intervention.

It has the additional advantage that the date of a photographic printnarrowed down to a date range when a particular type of photographicmaterial was manufactured by analyzing scans of watermark patterns onthe back of photographic prints.

It has the further advantage the cross-correlation analysis processrepresents a robust method for identifying watermark patterns that isinsensitive to variations introduced in the media manufacturing andscanning process.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart illustrating the method of the present invention;

FIG. 2 is an illustration of the back side of a photographic printdepicting the manufacturer's media watermark pattern;

FIG. 3A is an illustration of the back side of a photographic printincluding an ink printed photofinishing process applied stamp includingthe date of image processing;

FIG. 3B is an illustration of the image side of a photographic printincluding a photographically exposed photofinishing process appliedgraphic including the date of image processing;

FIG. 3C is an illustration of the back side of a photographic printincluding a digitally printed photofinishing process applied indiciaincluding a camera recorded time and date of image capture, film roll IDnumber, and film roll frame number;

FIG. 4A is an illustration of the image side of a photographic printincluding a camera exposed indicia recorded by the film media depictingthe time and date of image capture;

FIG. 4B is an illustration of the image side of a photographic greetingcard including a photofinisher applied decorative graphic including auser selected text and date;

FIG. 4C is an illustration of the back side of a photographic printincluding a manufacturer's watermark and a user applied handwrittenindicia including a date;

FIG. 5A is an illustration of the image side and back side of aphotographic print including a photographically exposed photofinishingprocess applied graphic including the date of image processing and anunidentified manufacturer's watermark;

FIG. 5B is an illustration of the image side and back side of aphotographic print including a user applied handwritten indiciaincluding a date, an unidentified manufacturer's watermark, and a uniquemedia shape characteristic;

FIG. 5C is an illustration of the image side and back side of aphotographic print including identified manufacturer's watermark and anidentified media shape characteristic;

FIG. 6A is an illustration of the back side of a photographic printincluding an identified manufacturer's watermark with the known timeperiod of manufacture “A”;

FIG. 6B is an illustration of the back side of a photographic printincluding an identified manufacturer's watermark with the known timeperiod of manufacture “B”;

FIG. 6C is an illustration of the back side of a photographic printincluding an identified manufacturer's watermark with the known timeperiod of manufacture “C”;

FIG. 7 illustrates a series of entries in a photographic print watermarkdatabase;

FIG. 8 illustrates cropping a watermark bitmap from a scanned referencepattern;

FIG. 9A illustrates translating a watermark bitmap across a matchingback scan;

FIG. 9B illustrates a cross-correlation signal for a matching watermarkbitmap;

FIG. 9C illustrates translating a watermark bitmap across a non-matchingback scan;

FIG. 9D illustrates a cross-correlation signal for a non-matchingwatermark bitmap; and

FIG. 10 illustrates an example watermark data structure.

DETAILED DESCRIPTION OF THE INVENTION

There are many applications where it is desirable to digitizephotographic prints. For example, a user may have a shoebox full offamily photographs and would like to make a digital database of thesephotographs for viewing and sharing with other family members. Oftenthese photographs are unlabeled, and therefore the user may have noinformation about the date that the photographs were taken. It isdesirable to be able to determine the date of the photographs forhistorical purposes, as well as to aid in the organization of thedigitized images. Manufacturers of photographic print materials oftenincluded watermark patterns on the back of photographic prints. Theterminology watermark patterns as used in the present application refersto visible patterns that are placed on the back of photographic printmaterials during the manufacturing process. Such watermark patterns aretypically printed on the photographic print materials using some form ofprinting process, such as offset or gravure printing. Often thewatermark patterns contain manufacturer-specific content such as companynames, logos or trademarks. Watermark patterns may also contain othertypes of content such as lines and various geometric patterns. Commonly,the watermark patterns are printed in light-colored inks, such as lightyellow, light blue or light gray inks, to minimize the likelihood thatthe watermark patterns will show through to be visible from the frontside a photographic print. These watermark patterns were often modifiedwith each succeeding generation of the photographic print materials, andtherefore provide valuable information about the date of thephotographic print. In U.S. Patent Application Publication 2006/0198559,Manico et al. have disclosed a method for automatically organizingprints based on analysis of the photographic print watermark patterns,the disclosure of which is incorporated herein by reference. The presentinvention is directed toward the analysis of photographic printwatermark patterns using a cross-correlation algorithm as part of theprocess of estimating the date of a particular photographic image.

The present invention will now be described with reference to FIG. 1.First a scan print step 10 is used to scan the front and back sides of aphotographic print 8 to produce an image scan 12 and a back scan 14,respectively. The photographic print 8 may be an optical photographicprint produced by printing a photographic negative on conventionalsilver halide photographic print materials. The photographic print 8 mayalso be a digital photographic print produced by printing a photographicimage on conventional silver halide photographic print materials using adigital printer. Alternately, it could be a photographic print producedusing other printing technologies such as thermal dye diffusion printersor inkjet printers. Next an identify watermark step 16 is used tocompare the back scan 14 with a photographic print watermark database 18to identify a matching photographic print watermark 20. Eachphotographic print watermark in the photographic print watermarkdatabase 18 includes a representation of a watermark pattern togetherwith an associated date range. A determine date range step 22 is used todetermine a date range 24 associated with the matching photographicprint watermark 20 responsive to the matching photographic printwatermark 20. In one embodiment of the present invention, the date range24 is determined by looking up the date range associated with thematching photographic print watermark 20 in the photographic printwatermark database 18. Finally an associate date range with image step26 is used to associate the date range 24 with the image scan 12 toproduce a dated image 28.

Several optional features are also illustrated in FIG. 1 using brokenlines. In particular, an analyze scan step 30 can be used to analyze theimage scan 12 or the back scan 14 to determine additional information 32that can be useful in the determining date range step 22.

Each of the features of the present invention will now be described inmore detail. In a preferred embodiment of the present invention, thescan print step 10 is performed using a high speed scanner, such as oneof the Kodak i600 Series Scanners, that simultaneously scans both thefront side and the back side of the photographic print 8. Such scannerscan be used in photographic kiosk systems for providing digitizationservices for collections of a user's photographic prints. Alternately,the scan print step 10 can be performed using other types of scanners,such as consumer flatbed scanners. In such cases, it would be necessaryto scan the front of the photographic print 8 to produce the image scan12 and then to physically turn the photographic print 8 over to scan theback of the photographic print 8 to produce the back scan 14.

FIG. 2 is an illustration of a back side 70 of a photographic print 60having an image on the front side (not shown). In the embodimentillustrated, the back side 70 has manufacturer's media watermarkpatterns 80 and 90 thereon. These watermark patterns represent awatermark style that would be recognized by the system by comparison tothe photographic print watermark database 18 (FIG. 1). The example shownin FIG. 2 illustrates a common configuration for watermark patternpresentation which includes lines of evenly spaced text and/or graphicsthat run diagonally across the back surface of hardcopy imaging media.In the embodiment illustrated, each of the watermark patterns 80 and 90comprise the repeating text “Kodak PAPER”. Watermark pattern 90 includesa watermark encoding mark 100 which, in the embodiment illustrated,appears as a short line over the “E” in the text “PAPER”. This encodingmark 100 is typically associated with a date of manufacture, or aparticular position on the master roll from which the print paper wasobtained.

The image scan 12 and the back scan 14 often contain additional clues asto the date of a photographic print. FIGS. 3A, 3B, and 3C illustratevarious photofinisher applied markings that are used by photofinishersto provide additional information about the photographic print 60. FIG.3A is an illustration of the back side 70 of a photographic print 60that includes a stamp 110 that is applied during the photofinishingprocess. In the illustrated embodiment the stamp 110 includes anoriginal date of processing indication 120 and additional information130 such as a logo, graphic, code number, and the like, either or bothbeing presented in eye or machine readable form. Also recorded on backside 70 is the manufacturer's watermark pattern 80.

FIG. 3B is an illustration of the image side 160 of photographic print60. In addition to the image 162, the image side 160 also includes aphotographically exposed photofinishing process applied graphic 150. Inthe embodiment illustrated, graphic 150 is printed in the border 140 andincludes the date of processing.

FIG. 3C is an illustration of the back side 70 of a photographic print60 that includes a printed indicia 170 that is applied during thephotofinishing process. In the illustrated embodiment, the printedindicia 170 includes an image capture time 180 and date 190, film rollID number 200, and film roll frame number 210. Information recorded byprinted indicia 170 is indicative of images captured using AdvancedPhoto System cameras.

FIGS. 4A, 4B, and 4C illustrate additional methods used byphotofinishers and photographers to provide additional information onphotographic prints. FIG. 4A is an illustration of the image side 160 ofa photographic print 60 including a camera exposed indicia 215 recordedon the photographic negative depicting the time 220 and date 230 ofimage capture.

FIG. 4B is an illustration of the image side 160 of a photographicgreeting card 65 including a photofinisher applied decorative graphic240, a user selected event designation 250, personal message 260, anddate 270. Recognizing the photographic print as a greeting card willprovide additional opportunity for event dating.

FIG. 4C is an illustration of the back side 70 of a photographic print60 having a manufacturer's watermark pattern 80 and a user appliedhandwritten date annotation 280. It is a common practice for users toannotate the surface of photographs with pens, markers, pencils and thelike to record significant information regarding events, date, personsand locations that are relevant to the images recorded on the media.Handwriting analysis and OCR software can be used to translate the textto usable data format.

FIGS. 5A, 5B, and 5C illustrate additional characteristics ofphotographic prints that can be used to provide additional informationabout the corresponding image scans. FIG. 5A is an illustration of theimage side 160 and back side 70 of a photographic print 60. In additionto a manufacturer's watermark pattern 290 recorded on back side 70, theimage side 160 includes photographically exposed photofinishing processapplied graphic 150 that includes the date of image processing printedin image border 140.

FIG. 5B is an illustration of a photographic print media 300 havingunique serrated media shape characteristics. The image side 160 includesborder 140. Back side 70 includes a handwritten date annotation 280, anadditional handwritten annotation 285 and a manufacturer's watermarkpattern 290.

FIG. 5C is an illustration of image side 160 and back side 70 for aphotographic print 60 with a media shape having rounded corners 305 witha ½″ (1.27 cm) radius, and a manufacturer's watermark pattern 80.

FIGS. 6A, 6B, and 6C illustrate various different types of watermarkpatterns used by manufacturers that have been made available to themarket over time. FIG. 6A is an illustration of the back side 70 of aphotographic print 60 including a manufacturer's watermark pattern 80with the text “Kodak PAPER.” FIG. 6B is an illustration of the back side70 of a photographic print 60 having a particular watermark pattern 310that was designed to commemorate a manufacturer's sponsorship of theOlympics and includes text and graphic. Watermark pattern 310 would beassociated with a fairly narrow date range. FIG. 6C is an illustrationof the back side 70 of a photographic print 60 having a particularwatermark pattern 320 that was designed by the manufacturer to indicatethe high quality, superior gloss and heavier stock of the media bearingthis watermark pattern.

Returning now to a discussion of the details of FIG. 1, the photographicprint watermark database 18 stores information regarding variousphotographic print watermarks that have been used by manufacturers ofphotographic print materials at different dates. The entry for eachphotographic print watermark in the photographic print watermarkdatabase 18 includes a representation of the watermark pattern togetherwith an associated date range. FIG. 7 shows a series of entries in aphotographic print watermark database 18 for a number of differentphotographic print materials manufactured by Eastman Kodak Company. Eachphotographic print watermark shown in FIG. 7 has associated with it adate range corresponding to the years that photographic print materialswith these photographic print watermarks were manufactured. Firstwatermark pattern 410 has an associated first date range 425. Likewise,second watermark pattern 420 has an associated second date range 415 andthird watermark pattern 430 has an associated third date range 435. Anactual photographic print watermark database 18 could potentiallycontain hundreds of watermark patterns.

There are a number of different ways that representations of thewatermark patterns can be stored in the photographic print watermarkdatabase 18 (FIG. 1). In a preferred embodiment of the presentinvention, a bitmap image is stored corresponding to the watermarkpattern. The bitmap image for a particular watermark pattern can bedetermined by scanning a sample of a reference photographic print havingthat watermark, and then cropping out a region of the scanned watermarkpattern. This is illustrated in FIG. 8 which shows a watermark bitmap440 cropped out of a back scan 442 for a particular watermark pattern.

Typically, watermark patterns that have been used by photographicmaterial manufacturers are periodic in nature, repeating on some fixedinterval. In such cases, it will generally be desirable to crop out awatermark bitmap 440 that contains at least one period of the watermarkpattern. In some cases, it may be desirable to process the watermarkbitmap 440 using a series of enhancement operations to provide a clearrepresentation of the watermark pattern. For example, it may bedesirable to increase the contrast of the watermark pattern 440 using ahistogram stretching algorithm or a thresholding operation. It may alsodesirable to correct for any non-uniformities in the background of thewatermark bitmap 440 to provide a uniform white background. Suchnon-uniformities could, for example, be caused by yellowing of thephotographic print material. It may also be desirable to perform a noisecleaning operation to remove artifacts resulting from dust specks,scratches and any other extraneous markings. The watermark bitmap 440can be stored as a continuous tone color image (e.g.,8-bits/pixel/color), or can be converted to grayscale and stored as acontinuous tone grayscale image (e.g., 8-bits/pixel) or a binarygrayscale image (e.g., 1 bit/pixel).

In another embodiment of the present invention, the representations ofthe watermark patterns can be stored in the photographic print watermarkdatabase 18 using geometric descriptions. For example, a watermarkpattern can be described by specifying geometric parameters defining therelative positions and sizes for a series of lines, circles, textcharacters and other geometric objects that characterize the design ofthe watermark pattern. For example, some photographic print materialsare manufactured using watermarks having a series of diagonal lines of agiven angle, line width and spacing. These patterns can be representedby a list specifying geometric parameters describing each line in thewatermark pattern.

There are various ways that the identify watermark step 16 can identifythe matching photographic print watermark 20. In a preferred embodimentof the present invention, a method utilizing a cross-correlationoperation is used. In a cross-correlation operation, a first image istranslated horizontally and vertically across a second image, and theintegral of the product of the two images is calculated at eachposition. When the two images are similar and are properly aligned, ahigher correlation signal is obtained; when the two images aredissimilar or are not aligned a lower correlation signal is obtained.This process is illustrated in FIG. 9A, which shows a watermark bitmap450 being translated across a back scan 452.

Mathematically, the cross-correlation operation is given by thefollowing equation:

$\begin{matrix}{{C( {x,y} )} = {{{B( {x,y} )}*{W( {x,y} )}} = {\sum\limits_{x_{0} = 1}^{N_{x}}{\sum\limits_{y_{0} = 1}^{N_{y}}{{B( {{x + x_{0}},{y + y_{0}}} )}{W( {x_{0},y_{0}} )}}}}}} & (1)\end{matrix}$where C(x,y) is a cross correlation image, B(x,y) is a back scan image,W(x,y) is a watermark bitmap, N_(x) and N_(y) are the horizontal andvertical sizes of the watermark bitmap, respectively, x and y are therow and column indices, and x₀ and y₀ are loop indices. This calculationis repeated for each x and y position to determine a fullcross-correlation image. FIG. 9B shows a surface plot 454 of thecross-correlation image computed for the example given in FIG. 9A. Itcan be seen that there are large correlation peaks 456 corresponding tothe x and y positions where the watermark bitmap 50 lines up with thecorresponding watermark pattern in the back scan 452. A threshold 458can be applied to the cross correlation image. If any correlation peaks456 in the cross-correlation image exceed the threshold, as is the casein FIG. 9B, then the back scan 452 is identified as containing theparticular watermark pattern.

The particular value used for the threshold 458 is determined bytraining the process using known patterns. For optimal performance, itwill be desirable to use a custom threshold for each differentphotographic print watermark in the photographic print watermarkdatabase 18. In a preferred embodiment of the present invention, athreshold value for each photographic print watermark is stored in thephotographic print watermark database 18 together with a representationof the watermark pattern and the corresponding date range for which theparticular photographic print watermark was used. A watermark datastructure can be defined to store the information associated with eachphotographic print watermark in the photographic print watermarkdatabase 18. An example of a watermark data structure 470 that can beused in accordance with the present invention is shown in FIG. 10. Thewatermark data structure 470 is comprised of several fields. A watermarkpattern representation field 472 is used to store a representation ofthe watermark pattern. As described earlier, this can either be awatermark bitmap or some other representation such as a geometricdescription. A threshold field 474 is used to store a threshold valuethat is appropriate for the particular photographic print watermark. Anearliest usage date field 476 and a latest usage date field 477 is usedto store a date range associated with the photographic print watermark.A watermark pattern ID field 478 is also used to provide a uniqueidentifier for each photographic print watermark.

FIG. 9C illustrates the case where a watermark bitmap 460 iscross-correlated with a back scan 462 that has a watermark patterndifferent from that of the watermark bitmap 460. FIG. 9D shows a surfaceplot 464 of the cross-correlation image generated for the example shownin FIG. 9C. In this case, the surface plot 464 does not contain anylarge correlation peaks. Therefore, when the threshold 458 is applied tothe cross correlation image, no peaks exceeding the threshold 458 arefound, and therefore no matching photographic print watermark pattern isdetected.

The orientation that the photographic print 8 is fed into the scanner isarbitrary. Therefore, the orientation of the watermark pattern in theback scan 14 is unknown in general. In order to properly detect thewatermark pattern, the cross-correlation process can be executed for aparticular watermark pattern in each of four different orientations.Each orientation corresponds to a different rotation of the watermarkbitmap in 90 degree increments.

During the identify watermark step, the back scan 14 is compared to eachorientation of each watermark pattern until a matching watermark patternis detected. Once a matching watermark pattern is found, it is notnecessary to continue checking the remaining patterns and orientations.Therefore to improve computation efficiency, the comparison process canbe halted as soon as a matching pattern is found. To maximize thecomputation efficiency, the highest probability watermark patterns canbe checked first and the watermark patterns that are rarely encounteredrarely can be checked last. An automatic learning process can be used toadjust the order that the patterns are checked by adapting to thedistribution of watermarks that are encountered in the system. Whenscanning a set of photographic prints, it may be desirable to check thewatermark pattern identified for the previous photographic print firstbecause it is likely that the prints may be from same roll of film, orat least from a similar time period.

Another way that the computational efficiency can be improved is toperform a preliminary screening step to determine whether any watermarkis present in the back scan 14. Some photographic print materials do nothave a watermark. Therefore, if the back scan is found to be totallywhite, there is no value to perfuming the cross-correlation calculationswith each of the different watermark patterns. In this case, it may bepossible to estimate the date of the photographic print using other cuessuch as back printing and print geometry.

If the photographic print 8 is skewed slightly during the scanningprocess, this can interfere with the proper detection of the watermarkpattern during the cross-correlation process. Therefore, it will behelpful in some cases to use a de-skewing algorithm to correct for anyskew introduced to the back scan 14 during the scan print step 10. Suchde-skewing algorithms are well-known to one skilled in the art, andgenerally rely on determining a skew angle by detecting the edge of theprint in the image scan.

For the case where geometric descriptions are used to represent thephotographic print watermarks in the photographic print watermarkdatabase 18, several different approaches can be used for the identifywatermark step 16. In one embodiment, the geometric description for aparticular photographic print watermark can be used to form acorresponding watermark bitmap that can be used with thecross-correlation process described above. Alternatively, the back scancan be analyzed to determine geometric parameters that can be comparedwith the geometric descriptions of the photographic print watermarks.For example, if the photographic print watermark contains a series oflines having different angles, line widths and line spacings, the backscan 14 can be analyzed to determine the geometric parameters associatedwith each of the lines. The geometric parameters can then be compared tothe geometric descriptions for each of the watermarks in thephotographic print watermark database 18 to identify the matchingphotographic print watermark.

Another method that can be used for the identify watermark step 16 is touse so-called scale-invariant-feature-transform (SIFT) techniques. Onesuch SIFT technique is described in Lowe, D., “Distinctive Features fromScale-Invariant Keypoints,” International Journal of Computer Vision,vol. 60, no. 2, pp. 91-110, 2004. SIFT features acquired via a SIFTalgorithm are scale and rotation invariant. The SIFT technique describedby Lowe identifies salient points as locations of an image that can beaccurately identified regardless of scale of the image and regardless ofmany different transforms of the image (e.g., different scanorientations and resolutions). The SIFT technique uses a cascadefiltering approach to identify candidate salient points. The SIFTtechnique then performs a detailed fit to the nearby data for location,scale, and ratio of principal curvatures for each candidate salientpoint. The SIFT technique rejects candidate salient points that have lowcontrast and are poorly localized along an edge. Next, a local imagedescriptor (i.e., “feature”) is generated for each salient point. TheSIFT technique generates features that are orientation invariant byfirst computing the gradient magnitude and orientation of each imagesample in a region around the salient point. The SIFT technique thenaccumulates these samples into orientation histograms summarizing thecontent over a 4×4 region. Each histogram may have eight binsrepresenting different orientations resulting in a feature that isrepresented by a 128 (4×4×8) feature vector. Comparison of featuresacquired from a back scan image and features from an exemplar watermarkimage can be made using a nearest neighbor search algorithm. Determiningif an exemplar watermark image matches the watermark (if any) on a backscan image can be made based on the quantity of features matched. Oneskilled in the art will appreciate that other techniques may be used toidentify the salient points of an image and the features of the salientpoints and feature matching.

There may be cases where two watermarks differ only in the color of thewatermark pattern. In such cases, it may be desirable to also useinformation about the color of the watermark pattern in the back scan 14during the identify watermark step 18. The color of the watermarkpattern can be determined by examining the color values of the non-whiteregions in the back scan 14. The resulting color value can be comparedto a reference color for a particular watermark pattern that can bestored as a field in the watermark data structures in the photographicprint watermark database 18.

Once the identify watermark step 16 has been used to determine thematching photographic print watermark 20, the determine date range step22 is used to determine the corresponding date range 24. In a preferredembodiment of the present invention, the matching photographic printwatermark 20 is identified using the watermark pattern ID field storedwith each watermark pattern in the photographic print watermark database18. In this case, the determine date range step 22 can operate by simplylooking up the date range stored in the date range field 476 in thewatermark data structure 470 for the matching photographic printwatermark 20.

In some cases, the date range associated with a particular watermarkpattern may be relatively narrow, allowing the photographic print 8 tobe identified within a narrow date range. In other cases, the date rangeassociated with a particular watermark pattern may be relatively wide.Therefore, it may be valuable to be able to use additional pieces ofinformation to further narrow the date range. One source of suchadditional information can be data extracted from back printing on theback of the photographic print. As was discussed earlier, such backprinting includes information printed on the back of the photographicprint during the photofinishing process, or labels added to the back ofthe photographic print by a user. For example, the back printing addedby many photofinishers will contain the date that a particular print wasmade or that the image was captured. If the presence of back printing isdetected in the back scan 14, then an optical character recognition(OCR) algorithm can be applied to identify the alphanumeric charactersin the back printing. The alphanumeric characters can then be analyzedto extract a date string.

If the back scan 14 is determined to include labels added by a user(e.g., handwritten notations), the labels can be analyzed to determinewhether they contain any date information. Algorithms for extractingalphanumeric characters from handwritten text are well-known in the art,and are commonly used for applications such as mail sorting. Suchalgorithms can be applied to search for any strings of number andcharacters that may provide clues as to the date of the print.

Additional information that relates to the date range can also beobtained by analyzing the image scan 12. For example, imagecharacteristics such as color saturation of the image, size, shape andformat of the photographic print and the style of any borders present onthe image can all provide clues as to the date of the photographicprint. In some cases, there may be a date printed on photographic imagethat can be detected. For example, as was discussed earlier, somephotofinishing systems automatically printed a date field in the borderof the image during the photofinishing process. Additionally, somecameras have optional features which allowed a date to be automaticallyadded to the image at the time of image capture.

The determine date range step 22 can supplement the date rangeassociated with the matching photographic print watermark 20 with anyadditional information extracted from the image scan 12 or the back scan14 to provide a best estimate of the date range 24 for the photographicprint 8. For example, if a date field is identified either on the backof the photographic print as illustrated in FIGS. 3A, 3C and 4C, or inthe border of the photographic print as shown in FIG. 3B, or on thefront of the photographic print as shown in FIGS. 4A and 4B, then anoptical character recognition algorithm can be used to read the date.This can then be used to narrow down the date range, often to aparticular day or year. Alternatively, other types of information thatcan be gleaned from either the image scan 12 or the back scan 14 can beused to refine the estimated date range. For example, the distributionof colors in the image scan can be used to estimate the amount of imagefading that has occurred. This information can provide clues about theage of the photographic print. Similarly, other factors such as imagesize, image shape, image format and image border style can all provideadditional information that can be correlated to the age of thephotographic print.

In one embodiment of the present invention, date ranges can bedetermined for each of a plurality of such factors. The determine daterange step 22 can then determine a combined date range by overlappingeach of the individual date ranges. For example, if the identifiedmatching photographic print watermark 20 has an associated date range of1960-1970, and the photographic print has an image size that wascommonly used in the years 1965-1980, then a combined date range of1965-1970 can be inferred.

In another embodiment, the determine date range step 22 can use aprobabilistic approach to calculate a highest probability date range.This approach may be appropriate when considering factors, such as colordistribution, that are not associated with a well-defined date range.One probabilistic approach involves determining individual probabilitydistribution functions for the dates for each of a plurality of datepredictors. The individual probability distributions can then becombined to form an overall probability distribution function usingmethods that are well-known in the statistical arts. A date range canthen be determined that satisfies a certain specified confidence limit(e.g., 95% confidence limits).

Once the date range 24 has been determined, the associate date rangewith image step 26 is used to associate the date range 24 with the imagescan 12. In a preferred embodiment of the present invention the daterange 24 is stored as metadata in a digital image file used to store theimage scan 12. For example, if the image scan 12 is stored using thewell-known EXIF file format, the date range can be stored in a field ofthe EXIF file header.

In an alternate embodiment of the present invention, the date range canbe used to control the location that the digital image is stored in ahierarchical file system. For example, different “folders” can beprovided for each decade, and the image file can be automatically storedin the appropriate folder corresponding to the date range 24.

A computer program product can include one or more storage medium, forexample; magnetic storage media such as magnetic disk (such as a floppydisk) or magnetic tape; optical storage media such as optical disk,optical tape, or machine readable bar code; solid-state electronicstorage devices such as random access memory (RAM), or read-only memory(ROM); or any other physical device or media employed to store acomputer program having instructions for controlling one or morecomputers to practice the method according to the present invention.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

PARTS LIST  8. photographic print  10 scan print step  12 image scan  14back scan  16 identify watermark step  18 photographic print watermarkdatabase  20 matching photographic print watermark  22 determine daterange step  24 date range  26 associate date range with image step  28dated image  30 analyze scans step  32 additional information  60photographic print  65 photographic greeting card  70 back side  80watermark pattern  90 watermark pattern 100 encoding mark 110 stamp 120date of processing indication 130 additional information 140 border 150graphic 160 image side 162 image 170 printed indicia 180 time 190 date200 film roll ID number 210 film roll frame number 215 camera exposedindicia 220 time 230 date 240 decorative graphic 250 event designation260 personal message 270 date 280 handwritten date annotation 285handwritten annotation 290 watermark pattern 300 photographic printmedia 305 rounded corners 310 watermark pattern 320 watermark pattern410 first watermark pattern 415 first date range 420 second watermarkpattern 425 second date range 430 third watermark pattern 435 third daterange 440 watermark bitmap 442 back scan 450 watermark bitmap 452 backscan 454 surface plot 456 correlation peaks 458 threshold 460 watermarkbitmap 462 back scan 464 surface plot 470 watermark data structure 472watermark pattern representation field 474 threshold field 476 earliestusage date field 477 latest usage date field 478 watermark pattern IDfield

1. A method for dating photographic prints comprising: a) scanning afront side of a photographic print forming an image scan; b) scanning aback side of the photographic print forming a back scan; c) defining adatabase of photographic print watermarks, each photographic printwatermark comprising a representation of a watermark pattern and anassociated date range; d) analyzing the back scan using across-correlation process between the back scan and the watermarkpatterns in the database of photographic print watermarks to identify amatching photographic print watermark, and by further analyzing the backscan to determine a color of the photographic print watermark in; e)determining a date range for the image scan responsive to the identifiedmatching photographic print watermark and the determined color of thephotographic print watermark; and f) associating the determined daterange with the image scan; wherein a matching photographic printwatermark is identified when a signal generated by the cross-correlationprocess exceeds a predetermined watermark-dependent threshold valueassociated with each photographic print watermark, wherein a trainingprocess is used to determine the watermark-dependent threshold value foreach photographic print watermark.
 2. The method of claim 1 wherein thescanning steps are performed using a high speed scanner thatsimultaneously scans both the front side and the back side of thephotographic print.
 3. The method of claim 1 wherein the scanning stepsare performed using a flatbed scanner.
 4. The method of claim 1 whereinthe representations of the watermark patterns in the database ofphotographic print watermarks include bitmaps of the watermark patterns.5. The method of claim 4 wherein the bitmaps of the watermark patternsinclude at least one period of periodic watermark patterns.
 6. Themethod of claim 1 wherein the representations of the watermark patternsin the database of photographic print watermarks include geometricdescriptions of the watermark patterns.
 7. The method of claim 1 whereinthe associated date range corresponds to the range of dates that aphotographic print material having the associated watermark pattern wasmanufactured.
 8. The method of claim 1 wherein the step of analyzing theback scan further includes a preliminary screening process fordetermining whether the back scan contains a photographic printwatermark.
 9. The method of claim 1 further including the step ofanalyzing the back scan or the image scan to provide additionalinformation that is used in the step of determining the date range forthe image scan.
 10. The method of claim 9 where any back printingcontained in the back scan is analyzed to provide the additionalinformation.
 11. The method of claim 10 wherein the back printingincludes information printed during a photofinishing process.
 12. Themethod of claim 10 wherein the back printing includes labels added tothe back of the photographic print by a user.
 13. The method of claim 10wherein the additional information includes image size, image shape,image format, image border style, image color characteristics or datestamp.
 14. The method of claim 1, further including the step of storingthe image scan in a digital file.
 15. The method of claim 14 wherein theassociated date range is stored as metadata in the digital file.
 16. Themethod of claim 14 wherein the digital file is stored in a hierarchicalfile system in a location associated with a particular date range. 17.The method of claim 1 further including a de-skewing step to correct forany skew introduced into the back scan during the back side scanningstep.
 18. The method of claim 1 wherein the cross-correlation process isapplied for multiple orientations of the watermark patterns.
 19. Asystem for dating photographic prints comprising: a) a scanner forscanning a front side of a photographic print forming an image scan; b)a scanner for scanning a back side of the photographic print forming aback scan; c) a database of photographic print watermarks, eachphotographic print watermark comprising a representation of a watermarkpattern and an associated date range; d) a processor for analyzing theback scan using a cross-correlation process between the back scan andthe watermark patterns to identify a matching photographic printwatermark from the database of photographic print watermarks, analyzingthe back scan to determine a color of the photographic print watermark,and determining a date range responsive to the matching photographicprint watermark and the determined color of the photographic printwatermark, wherein a matching photographic print watermark is identifiedwhen a signal generated by the cross-correlation process exceeds apredetermined watermark-dependent threshold value associated with eachphotographic print watermark, wherein a training process is used todetermine the watermark-dependent threshold value for each photographicprint watermark; and e) a means for associating the date rangeassociated with the identified photographic print watermark with theimage scan.